The present disclosure relates to semiconductor circuits, and more particularly, to an intelligent high bandwidth memory appliance including a logic die for offloading complex logic operations traditionally performed by a host.
The Internet has caused a huge expansion in the number of computer servers that serve data to millions of computers and mobile devices. Artificial Intelligence (AI) and other deep learning applications are becoming more common and are presently in high demand. Today's server computer environment is moving toward in-storage and in-memory computation, so that some computation is performed closer to where the data actually resides. This increases performance and reduces energy consumption.
Emerging applications like deep neural networks need massive computational and memory abilities to train on different datasets and learn with high accuracy. Moreover, as applications like high performance computers (HPC), graphics algorithms, and the like, become data and compute intensive, energy-efficiency and low latency become critical.
Recently, High Bandwidth Memory (HBM) and High Bandwidth Memory 2 (HBM2) have been used to achieve higher bandwidth while using less power in a smaller form factor by stacking Dynamic Random Access Memory (DRAM) dies one atop another, and provide an asynchronous communication interface with a host. The asynchronous nature of the communications increase performance but also makes it more difficult to process complex logic operations. When logic operations are complex, there is less determinism. In other words, it is less certain how long a particular complex logic operation will take to complete.